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SUMMARY 

In this paper we describe certain spectrum transformation techniques that can be 
used to transform a diverging iteration into a converging one. We consider two 
techniques called spectrum scaling and spectrum enveloping and discuss how to obtain 
the optimum values of the transformation parameters. Numerical examples are given to 
show how this technique can be used to transform diverging iterations into converging 
ones; this technique can also be uBed to accelerate the convergence of otherwise 
convergent it er at ions . 


1. INTRODUCTION 


Consider the linear system 


Ax m b 


( 1 ) 


which may be rewritten as 


x = Tx = c 


( 2 ) 


and solved using the iteration 

x n+1 = Tx n + c (n^O) ( 3 ) 

This iteration, called the basic iteration, converges to the unique solution of 
equation (2) if and only if the spectral radius of the iteration matrix T is 
smaller than 1, i.e., p( T) < 1. 

When the basic iteration defined by equation (3) is convergent, a number of 
methods can be used to accelerate the rate of convergence. Such acceleration schemes 
are known as Chebyshev acceleration (ref. 7), semi-iterative methods (SIM) (refs. 4 
and 10), and hybrid SIM (ref. 3). Several important optimality results have been 
established in the cited papers. 

Now consider the case when the basic iteration described by equation (3) is 
divergent, i.e., when some eigenvalues of the iteration matrix T lie outside the 
unit circle (/?( T) > 1). In such cases, only a few techniques are known that allow 
for the solution of equation (2) to be obtained iteratively. In particular, when the 
real parts of all eigenvalues of the basic iteration matrix T lie in the interval 
(-1,1), it is possible to use the spectrum enveloping technique (refs. 2 and 8) to 
transform the basic iteration in such a way that the modified iteration is guaranteed 
to converge. When the real part of any eigenvalue of T lies outside the interval 



(-1,1) most of the acceleration schemes fail, except possibly for the hybrid SIM 
which has been shown to apply in certain special situations (ref. 3). 

We consider the situation where the eigenvalues of the iteration matrix T may 
lie anywhere in the left half of the complex plane, in fact, real {£") < 1, where 

f + irj € (7 ( T) , <7(T) = set of all eigenvalues of T. In this situation, a simple 
linear transformation can be used to map <7{T) onto S = {£ = £ + if): -1 < real (£) 

- £ < 1}. Once the transformed eigenvalue spectrum lies in S, an enveloping ellipse 
can be defined to obtain guaranteed convergence. The composite procedure is 
equivalent to a polynomial mapping of G(T) the unit circle. 

In this paper, we examine the conditions under which the spectrum scaling and 
spectrum enveloping transformations are successful. We determine the optimum values 
of the transformation parameters and show that the parameters of spectrum scaling may 
be chosen so as to obtain convergence. The spectrum scaling may also be used to 
accelerate the convergence when the basic iteration given by equation (3) is 
convergent. The spectrum enveloping transformation is used to obtain even faster 
convergence. We present examples to exhibit the theoretical and computational values 
of the transformation parameters and corresponding values of asymptotic convergence 
factors . 


2. THE SPECTRUM SCALING 


Consider the basic iteration: 

x n +1 = T x n + c, (n ;> 0), x 0 arbitrary ( 4 ) 

Assume that the eigenvalue spectrum of the iteration matrix T, denoted by G( T) , is 
a compact set in the complex plane C. Let £ = £ + L7] be an eigenvalue of T. The 
iteration (4) is convergent if p{ t) < 1, i.e., |£| 2 * | £ 2 + Q 2 ] < 1 for all £"6a(T). 
The iteration (4) diverges if |£"| > 1 for any £"e<7(T). 

Now consider the case when |£| >1. In particular, let 


-«» < a z Real ({) = % z A < 1 ( 5 ) 

< -b * Iwag {{) = T) s b < « 

i.e., <7(T) is contained in the rectangle [a,A]x[-b,b] in the complex plane. 

We define a two-step iteration 

z n*l = Tx n + C < 6 > 

= P z n*l + (1 " P>* a 

The combined form of equation (6) is given by 

x n*i = t' x n + pc ( 7 } 


2 


where 


(8) 


T' ■ pT + (1 ~ p) I 

The eigenvalues f (= + if) of T' are related to the eigenvalues £ of T 

through the following relations: 


C' - PC + (1 - P> 

l' = p5 + (i - p) , V = p*i 


The scaling parameter p must be chosen such that |£'| ^ 7 < 1 ^ or fCO’(T) 

and for some 76 ( 0 , 1 ) : 


-l<-p(' = p(S -1) + 1 * y < 1 


or, 


1 - Y 
i - 5 


s p s 



(9) 


As equation (9) must hold for all £ = £ + i f] r the value of the scaling parameter p 
must satisfy 


0 < 


1 - 1 
1 - A 


s p S 


1 + y 
1 - a 


( 10 ) 


This condition requires that 


0 < Y 0 = 



H <1 


(ID 


With conditions (10) and (11), the eigenvalue spectrum (T(T') of T' lies in S: 


= fj + itj' e o ( TO 

-1 < *Y s s Y < 1 

-oo < -b — — ~ z t\ ! z b - < 00 

1 - A 1 - A 


( 12 ) 


The spectral radius p( T') of the transformed matrix T' is given by p 2 ( T') 

= 7 2 + p 2 b 2 , which is the smallest when p takes its minimum permissible value 
(1 - 7 ) / ( 1 - A) . Thus, 

(1 -A) 2 

Theorem 1 . - The value of /) ( T ' ) given by equation (13) is minimum when 7 is 
chosen such that 7 = max ( 7 . 7 , ) , where 7 = (A - a)/(2 - A - a) and 
7i = k/(l + k), k = b 2 /(l - A) 2 . 
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Proof . - From equation (13), p 2 ( T') « 7 2 + k(l - 7 ) 2 , where k = b 2 / ( 1 - A) 2 > 0. 
This expression takes a local minimum when 7 = = ( 1 + k) • Consider two cases: 

If 7 n < 7, < 1/ the minimum value of /)(T') is clearly obtained when 7 - 7 ^ In 
this case, /J(T') = = V [k/(l + k) ] < 1. If 7j < 7 0 < 1/ the value of />( T') is 

an increasing function of 7 in ( 7 Q , 1 ) and the minimum value of /?( T # ) is obtained 
when 7 ® 7 • In either case, the minimum value of p( T') is obtained when 
7 = max( 7 Qf ° 7 X ). 

Remarks . - The spectrum scaling defined in this section is applicable whenever 
£ * Real(£) < 1 for all eigenvalues £ of the iteration matrix T. Similar scaling 
can be defined, and corresponding values of optimal parameters obtained, when £ > 1 
for all eigenvalues of T. 


3. THE SPECTRUM ENVELOPING 

If the eigenvalue spectrum of the iteration matrix T lies in the infinite set 
S {£: -1 < real(£") < 1} in the complex plane, irrespective of whether T defines a 
convergent or a divergent iteration, a spectrum enveloping transformation can be 
defined to obtain accelerated convergence (refs, 2 and 8). In this section, we 
consider the problem of obtaining the optimum values of the transformation 
parameters . 

Let the eigenvalues £ = £ + irj of the basic iteration matrix T satisfy the 
following conditions: 


-1 < -y s Real(0 = 5 5 y < 1 (14) 

-» < -p s Imag ( ( ) = i\ z P < <» 

Define an ellipse, lying entirely within S, with semi-axis of length M lying on 
the imaginary line and semi-axis of length m lying on the real line, 0 < m <1. The 
spectrum enveloping iteration for the basic iteration (3) may be written as follows: 

**♦1 = Ty n + c (15) 

y an = (1 + - W y a - 1 


where X - (m - M)/(m + M) , and fi is the unique root in (0, 1) of 
(m - M )fi 2 - 2fl + (m + M) =0. 

It is known (refs. 2 and 8) that the sequence {y} defined by equation (15) is 
convergent whenever X is nonzero; the asymptotic convergence factor is given by 


<Ho 


M + m 

1+v/T M2 - m2 + 1J 
M+m 


= 1 - 


Af+1 

1 -/ 7 ? 


as in < 1 


1 + M 


< 1 
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The semi-axes m, M of the enveloping ellipse must be chosen so as to contain 
all elements of the set CT(T); i.e., m, M must be chose n such that 

<y 2 /tn z + fj 2 /m z $ 1 . This requires that M > m/?/V [m 2 - "f 2 ) • As fl is an increasing 
function of M, we use the smallest value of M to define the enveloping ellipse: 

M = (16) 

'f[m L - Y 2 ] 


We now have, 

H < l » 0 = 1 - f (m) 


where 


f (m) 


1 - m 

1 + ^P ~ 

/ [m 2 - 


(17) 


In order to obtain the optimum convergence we minimize the value of fi Q ; this is 
equivalent to the problem of maximizing the value of f(m). 


Theorem 2. - f(m) given by equation (17) is maximum when m lies in the interval 


” 773 “ 

<7' 7 ) 


Proof . - Note that f ( 7 ) = f ( 1 ) = 0 and f(m) > 0 for m 6 ( 7 , 1). The derivative 
of f(m) is given by 


f'(m) = 


-1 + P(V 2 ~ M 3 ) 

(m* - r p ) 3/2 

A 2 


(18) 


2/3 

where A is the denominator in equation (17). As f'(7) > 0, anc * f'(7 ) 2/ < the 

peak in the graph of f(m) is obtained somewhere between m = 7 and m « 7 

The next result describes an optimum value of m. 


Theorem 3 . - 
of the equation 


An optimum value of the semi-axis m is given by the positive root 
(1 + P)m 3 - 1.5 y 2 m - Py 2 =0 


Proof. - The maximum value of the function f(m) is obtained at the point where 
f'(m) = 0. From equation (18), f'(m) = 0 when 

P (y 2 - m 3 ) = (m 2 - y 2 ) 3/2 
which gives, as a first approximation, 

P (y a - m 3 ) = m 3 ^l - j 


which yields the cubic equation (19). 
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From Theorem 3, the positive root of equation (19) gives the optimum value of m: 

when D = ( 7/2 ) 4 [ 2/? 2 ( 1 + /?) - ] / { 1 + P) 3 £ 0. The above root of equation (19) is 

valid when p is large enough so that D £ 0. In this case, the value of m may 
often be approximated by 

- «% - ,ai> 

When P is small so that 7 s > [2/J z (l +/?)], the three (real) roots of equation (19) 
are given by 


m 




p) 


J (1 + p) 
^ ] 


cos 

±, 

3 

cos 

4> + x 

3 

cos 

<J> ~ 7t 

3 


( 22 ) 


where cos(^) « 'y~ 1 pj~[2 + 2/?] . In this case, the optimum value of m is given by 
the root in equation (22) that lies in the interval ( 7 , 7 2/3 ) . If no such value of 
m is available from equation ( 22 ), the length of the semi-axis m is taken as 
7 2/3 . In either case, the value of m given by equations ( 20 ) and ( 22 ) provides a 
near optimum value of the asymptotic convergence factor p < fl = 1 - f(m) and the 
asymptotic rate of convergence is given by -log fi > -log fl Q . 


4. THE COMPOSITE TRANSFORMATION 

The composite transformation is obtained by first applying the spectrum scaling 
to the iteration given in equation (3), and then applying the spectrum enveloping to 
the scaled iteration given by equation (7). The composite iteration is described as 
follows : 

y n>1 = p( 1 + Aji 2 ) (7y ml + c) + (1 - p) (1 + A|i 2 )y n - A (i 2 y n .! < 23 ) 


This is a stationary two-step iteration. As all optimized semi-implicit methods for 
equation ( 2 ) degenerate into stationary two-step methods on computers with finite 
word length (ref. 7), we need consider only two-step iterations. This iteration 
involves the parameters p, 7 , m, and M defined by 


0 < fjir i-Hi, 

1 - A 1 - a 


0 < Y 0 


A - a 
2 - A - a 


* Y <1, 


A = HL — ** 0 < y < m < 1 , M = 

m + M 


mpb 

- y 2 ) 
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The optimum values of these parameters are given by: 


(24a) 


Y = maxi y 0 , Yi) 


where 

7o = (A - a) /( 2 - A - a) 

'j 1 = k/(l + k), k = b z /(l - A) 

p = 1 ~ Y . (24b) 

* 1 - A 


A. = 


m - M 
m + M 


(24c) 


M 




/n 2 - y z 


(24d) 


where m is given by Theorem 3. The asymptotic convergence factor fi for iteration 
(23) is given by 




Af + m 


1 + ^A£2 - m 2 + 1 


< 1 


The asymptotic rate of convergence is -log/i > -log/3 (T) 


5. THE COMPUTATIONAL ALGORITHM 

When the basic iteration defined by equation (3) is divergent under the 
conditions given by equation (5), it is often a simple matter to estimate the 
dominant eigenvalue of the iteration matrix T using the power method (ref. 12). 

This eigenvalue may provide a good estimate for a and b; the value of A may be 
estimated through the use of the power method on the matrix (T - al). Of course, 
more sophisticated methods are available in the literature for estimating the extreme 
eigenvalues of T (see, e.g., (refs. 6 and 11)). 

Once the values of a, A, and b are available, the scaling parameter p, the 
scaled matrix T', and the optimum scaling interval (- 7 , 7 ) may be obtained from 
Section 2. If the eigenvalues of T satisfy the conditions (14) of Section 3, or if 
it is desired to apply the spectrum enveloping to the scaled iteration defined by 
equation (7), the parameters m, M of the enveloping ellipse may be obtained from 
Section 3 and convergence follows. 

The composite iteration described by equation (23) may be carried out in three 
phases : 

Phase 0: Compute using equation (23) with p = 1, X = 0 to estimate a, A, b. 

Phase Is Obtain scaling parameters p and X from Theorem 1 and compute 

using spectrum scaling with equation (7) (or eq. (23) with X = 0) . 
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Phase 2; 


Obtain enveloping parameters m, M, X, and ji from Theorems 2 and 3 
and compute using spectrum enveloping with equation (15) (or eq. (23) 
with p = 1 ) . 

Phase 3: Obtain scaling and enveloping parameters p, 7 , X, ji, m, and M from 
equation (24) and compute using the composite iteration (23). 


6 . NUMERICAL EXAMPLES 


We now consider a few examples. 


Example 1 : Consider the following convection diffusion equation representing 

convection dominated flows: 

-J— + — ) + |H = °, 0 < x < 1 , 0 < y < 1 

V x z y 2 ) ox 

with appropriate boundary conditions (refs. 5 and 8 ). 


This differential equation is discretized on a uniform (N + 1) by (N + 1) grid 
for 6 = 0*005 using central difference approximations. When the resulting linear 
system of order (N - l ) 2 is solved using Jacobi iteration, the iterations diverge 
for N £ 32 (ref. 8 ). The eigenvalue spectrum of the Jacobi iteration matrix lies in 
the set S: -1 < real (£) = £ + LfJ <1 for all eigenvalues an enveloping ellipse 

may be defined to obtain convergence. As an example, when N » 8 , the eigenvalues £* 
of the iteration matrix T satisfy -7 £ £ i 7 ' ~P ^ V ^ P with 7 = 0.46194, 

P - 5.75574, and p( T) = 5.7742 (fig. 1). The parameters for the enveloping ellipse 
in phase 2 may be obtained from equation (21) as m = 0.5665 and M = 9.943266 and 
the asymptotic convergence factor is given by ji - 0.957404. 

The eigenvalue spectrum of the Gauss-Seidel iteration matrix lies outside the set 
S; the conditions (5) are satisfied with a » -33.1385, A = 0.2134, b * 5.3176, 
p( T) * 33.3419 (fig. 2). Optimum values of the scaling parameters p, 7 are 
obtained from Theorem 1: 7 - max(7 Q , 7 x ) = max(0. 954955, 0.978587) = 0,978587 and 

p as (l - 7 )/(l - A) = 0.027222. The scaled matrix T ' of phase 1 satisfies 

p(T') « VT^V] = 0.989235 < 1 (fig. 3). The use of spectrum enveloping of 
phase 2 provides the parameters for the enveloping ellipse: m * 0.98568, 

M = 1.208813 which provides even faster convergence with the asymptotic convergence 
factor ji = 0.988280. 


Example 2 : Consider the biharmonic equation: 


+ 2d*u + 8*u _ 0 
dx 4 dx 2 dy 2 3y 4 


0 < x < 1 , 


0 < y < 1 


with prescribed boundary conditions. This equation is discretized on a uniform 
(N + 1) by <N + 1) grid using the standard 13-point finite difference approximation 
(refs. 1 and 9). The resulting linear system of order (N - l ) 2 may be solved 
iteratively. 


When Jacobi iteration is used to solve the resulting linear system, the 
iterations are found to be divergent for all values of N and the spectral radius of 
the iteration matrix is found to be close to 2. As an example, for N = 8 , 
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a = -1.9699, A = 0.9454, b = 0.02927, and p{ T) = 1.9699 (fig. 4). The spectrum 
scaling is applicable with the optimum values of scaling parameters given by 
Theorem Is 7 = 0.9639 and p = 0.6613. The resulting iteration of phase 1 is 
convergent with p{ T' ) = 0.9641 (fig. 5). When the spectrum enveloping of phase 2 is 
applied to the transformed iteration given by T', the optimum values of the 
enveloping ellipse are given by m = 0.98205 and M = 0.101171 and the composite 
iteration is convergent with the asymptotic convergence factor p = 0.892256. 

When Gauss-Seidel iteration is used to solve the linear system corresponding to 
discrete biharmonic equation, the iterations are convergent, e.g., for N = 8 , p( T) = 

0.901248. Spectrum scaling of phase 1 may still be carried out: For N ■ 8 , 

a = -0.001816, A = 0.898199, b « 0.074083 (fig. 6 ). The spectrum scaling is 
applicable with the scaling parameters 7 = 0.815514, p = 1.81222, an^ p{ T' ) = 
0.826491. The composite use of spectrum scaling and spectrum enveloping in phase 3 
gives ft = pb = = 0.134255, m « 0.872879, and M = 0.376549 and the composite 
iteration defined by equation (23) converges with the asymptotic convergence factor 
jl = 0.7730. 


CONCLUSIONS 

We have described two spectrum transformation techniques, viz., spectrum scaling 
and spectrum enveloping, that can be used to transform a diverging iteration into a 
converging one. We have also discussed the method of obtaining the optimum values of 
the transformation parameters. Numerical examples show that, in addition to their 
effectiveness on divergent iterations, these techniques are also useful for 
accelerating the convergence of an already converging iteration. 
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Figure 1: Jacobi eigenvalues a(T) of CDS matrix 
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Figure 6: Gauss-Seidel eigenvalues a(T) of biharmonic matrix 
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